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ABSTRACT ♦ ' 

Although unreliable and not predictive of nedicaL 
school perfor»aace, the a dmis«ion. int er view continues to be usei 
extensively to collect noncognxtiVe Jata about tedical school 
applicants* A procedure , i? sugqested for assurincf that a group of 
interviewers assigned to an aptslicant does not unfairly Relp or 
hinder the candyiate's ratlna.-^. At least 3 medical school faculty 
interviewed and^ted 231 af>plici'^nts as exceptional-, acceptable, 
minimal'ly acceptable, or nnsAiitable: reliability between raters was 
low. To minimize this problem, a difficulty*cons4:stency index wss 
developed for classifying interviewers, and » rule-of -thu rab was 
proposed for , assuring fairness in assigning interviewers. Data from 
one year's Applicants show that-UB of 1UB candidates who were 
actually addWLtted to the cIp'ss of 1 93 3 , were inf.erviewed by a group 
^of interviewers wi^th an unsatisfactory diff iculty-consistency . total, 
(Author/CP) ♦ ,^ 
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k Th« medical school admission inttrvitw has ba^n shown to ^corralata 

poorly^with performance in medical school (1-7). In addition to lack of* 

« 

evidence for predictive Validity, critfcs have noted that medical\_scho6l 
admission intervlj|w8 are unreliable (8) and expensive in terms of pro- 
fessional time (9) . 

Nevertheless, the interview continues to be widely used. Poorman 
(10) reported that the 1975-76 Medical School Admission Requirements 
Handboo jj c indicated that lOA^''^' 109v U.S. medical schools providing data 
uti lized thu" interview: in~ toiaglr^ (-7) observed " 

that pirbponents o^ the medical schpol admission interview support its 

/ ■ ■ . • ■ . ■ 

use/bn two different unverified bases . First, the interview can be used 

to id(fent.ify an^' reject applicants "for reasons which cannot be discerned 
through examination of data submitted by applicants. Second, the inter- 
view to some extent assures the applicant that he is receiving individual 
attention. It is the authors' view that other means (i.e., structured 
personality tests, letters of recommendations, essays) of assessing non- 
cognitive information (e.g., personality characteristics; attitudes, 
motivation, interests) have not proven satisfactory. Lacking a viable 
substitute, the iriteryiew has p,ersisted in llarge part because of the 
appeal of personal involvement. Whatever the justification, Poorman (10) 
is no doubt correct when he states that in medical schools "the interview 
is here to stay; if not by reason, then by tVadition." .(P-301) 
Problem , \ 

Since the interview will remain an important part of the medical 
school admission process despite its drawbacks, what can be done to 
assure fairness *in its use? Medical School admission commi*ttees have 
expr-essed concern that »an applicant's chances of being selected often 
depend uppn the interviewers to whom he is assigned and not necessajrily 

9^- . . 3 



upon th« noncognitive criteria whipl^ the interview is ta assess. 

This study addresses the issue of fairness by' examining two im- 
portant components of interview fairness - the diffic<f?.ty and consis- 
tency of interviewers - in the cohtiaxt of one medical school's admission 
process] Difficulty refers tp the tendency of an interviewer to assign 
higheV or lower applicant scores ixx comparison with other interviewers. 
Consistency refers to the degree of agreement among interviewers rating 

' r 

the s^me applicant. 



Setting 

The Texas College of Osteopathic Medicine admits its entering class 

by assessing applicants on a variety of criteri^; premedical grade 

point average, scores on the Medical College Admission Test (MCAT) , 

» 

quality of experience in health-related work, exposure to and under- • 
standing of the osteopathic profession, likelihood of fulfilling the 
college'e mission of . providing general practitioners for the state of 
Texas, premedical academic and other honors, and suitability for the 
practice of medicine as judged by letters of, recommendation and one-on- 
one admission interviews. 

An Admission Committee establishes substantive areas in whiqh 
interviewers are to judge applicants. F^pr th^ class Aent.e ring in the 
fall of 1979 (Class of 1983) these areas were pr6blem-solving , life 
problem-solving, human interactions, responsibility, sotiial sensitivity 
and awareness, osteopathic motivation, and self-appraisal. The faculty 
interviewers participated in a September 1978 workshop prior to the 
October to March interviewing period. The workshop stressed the sub- 
stantive areas mentioned above, techniques for interviewing, anji trial 
as'sess.ment^ , * 

•» 

' - 2 - 



Data ' 

^ During Che five-month interview period 231 app>l;^cant8 wer« invite" 
for interviews. , Each aptflicant was interviewed individually by at least 
three faculty members. The applicant waa., rated either exce^btional (3), 
acceptable (2), minimally acceptable (1), or reject (0). In 24 of the 
708 interviews rater's, chose a rating between two categories (e.g., 1.5 - 
between m*inimally acceptable and acceptable) . Of the 231 applicants 
interviewed, 216 received three interviews while /15' received an addi- 
tional or fourth interview. - The fourtH interview wtt« employed when an 
applicant ^received an exceptional (3) and reject (0) rating among^ his 
original thr«e ratings .» ^ ' 

Table 1 presents data related to the 30 faculty interviewers who 
conducted 10 or more interviews. Fifteen faculty interviewers who con- 
ducted le^s than 10 interviews were excluded'^f rom the study. 

The columns headed group refer to the mean and standard deviation 
for the rotating^ g^,o^P of two or three colleagues who rated the same 
applicants as the interviewer. The mean absolute difference is the 
average difference between the interviewer and the group rating. The 
mean +/t difference is an index of aR interviewer's tendency to be a 
difficult rater (minus) or kn easy rater (plua) . correlation co- 

efficient' is t'earson r between interviewer and group ratings. 
Results and Discussion 

Inspection of the interviewer mean column in Table 1 reveals that 
an applicant's total rating could vary dramatically depending on the 
interviewers he was assigned. (Interviewers were assigned on a con- 
venience basis and not with a random procedure.) While three "average" 
interviewers would result in a total ratings of 5.52 (1.84 x 3), other 
combinations would result in very different totals. If rating an "average' 
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applicant,, tht three most difficult interviewers (nos . 5, 21, and 30) 
would yield a total of 3-94 (1.38 + 1.33 + 1.23). The three easiest 
interviewers (nos. 14, 23, and 25) would yield a total of 7.19 (2.50 + 
2,41 + 2.28). Based gn the cutoff level ^established by the Admission 
Committee, the latter ^applicant would be a viable candidate fo» ad- 
mission, the former would not. 

Thus, given that the tatings are of questionable reliabilit?y , how' 
can fairness be assured in circumstances where the\ i^iterview process 
must utilize a cadre of interviewers who ar^ not professionally trained 
for i-nterviewing and whose academic heterogeneity and personal' perspec- 
tives on the definit;^ion of a' physician differ greatly? 

One often sViggested solution to the' problem outlined above is to 
adjust each interviewer's mean rating in accordance with the group mean. 
Thi>s , interviewer no. 4 might have . 07 'subtracted from each rating since 
he is .07 above the group mean "(1.70 - 1.63). . This procedure assumes 
that applicants are randomly/ assigned to interviewers. The assumption 
is made that all interviewe'rs in the long run are given applicants of 
equal merit to interview. This, of course, is not the case. The 
metiical school interview^ process is reliant upon the scheduling con- 
tirtgeHc|^es of both interviewers and applicamts. 

An alternative procedure which is -le^s dependent on the assumption 
of random assignment Jises a two-variable approach to categorize inter- 
viewers. Table 2 categorizes the 30 interviewers on Che dimensions of 
difficulty and consistency. Rational judgmen^t was used by the authors 
in establishing three categories of difficulty and consistency. (I^te: 
Statistical te3ts to ascertain differences in difficulty 'arVd consistency 
on the basis of sex, §ge , and academic department were nonsignificant.) 



Table 2: Dif f icuity-Consisttncy Classif icttion of Interviewers* 



>, 1. 60 or 
^ ' less 

.2 • 1.61-2.05 

5 more than 
° ^ ' 2.05 



Consistency (Pearson r) 

less than 

.30 .30' - .49 



more than 
.49 



5 ■ 2 -A 


3 • ' D . 


1, 2 G 


4 * ■ B 


2 ' E 

> /■ 


1 ^ H 


5 ' C 


3 ^ , F 





Cells G, H, and I would be assigned a, difficulty-consistency .index* 
of 1; cell E would be assigned an index of 2; cells D and F would be 3; 
cell B would be 4; and cells A and C would be 5. On the. 1-5 continuum 
a 1 represents the most desirable^ interviewers (i.e., those most in 
agreement with their group of interviewers) while ^ 5 r||presents "the 
least desirable ' (i. e. , those that are either difficult and not consistent 
with their group or easy and not cons.istj^nt with their group) . 

An admission committee might be advised to establish a rule-of -thumb 
by which an applicant would not be interviewed by lihree interviewers who 
total" to more than eight.' Thus, if an applicant were assigned a 5 (easy 
or difficult and inconsistent), he would also have to be assigned' a 2 and 
a 1 or two I's, that is, two of the better interviewers. 



^Table 2 should be read as follows 



difficulty -consistency 
index 




no 



of 'interviewers 
in cell 



cell label 
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If this rul«-of- thumb were applied retrospectively to the Clajs of 
1983 interviewers, how m&rxy applicants* would have been "fairly" treated 
(i.e., assigned a group' of interviewers whose difficulty-consistency" 
classification totalled to 8 or less) . Table 3 report^ these data. 



Table 3; Number of Applicants in Each Dif f iculty-^Consis tency 

Classification 



No. of 
Applicants 



. Qif f iculty-Conjsistency Classification 
For Interview Groups* 



. 3 



8 



10 11 12 13 14 15 



15 14 15 20 19 17 13 6 10 14 3 



Mean 
S.D. 

N 



7.34 
2.93 
148 



0 



> One hundred of 148 '(67.6%) applicants were interviewed by a group of 
three interviewers whose, difficulty-consistency classification was 8 
or less. Forty-eight (^2.4%) applicants were interviewed by groups 
who totalled 9 'or more. Thus, nearly one-third of 'the Class of 1983 
applicants was- interviewed by a group of interviewers whose difficulty- 
consistency total was unsatisfactory. ^ 



■^Of the 231 -applicants^ , 15 wei^e not included in Table 3 in that they had 
four interviews and ^8 were not included in that they had oi^e or more 
interviews by the least active interviewers not included in .Table 1. 
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Although unreliable and n'd't predictive of medical school performance , - 
the admission interview continues to be used extensively as a means of 
collecting noncognitive data aboiit medical school applicants, this study 
suggests a procedure for assuring. that a group of interviewers assigned to 
an applicant does not 'unfairly help or hinder the candidate ' s -ratings . 
The difficulty-consistency index was developed for the purpose of classi- 
fying interviewers, and, a "rule-of-thumb proposed for assuring fairness in 



assigning interviewers. Data from one year ' s' applicants show that 48 of 
148 candidates were Interviewed by a group of interviewers with an un- 
satisfactory difficulty-consistency total. The procedure described was 
developed \i the practical setting of one medical school's admission 
process. It is suggested that continued practical research combined 
with experimental study of the fairness issue offers promise for im- 
provement of the much-maligned medical school admission interview. 
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